1. Field
Apparatuses and methods consistent with exemplary embodiments relate to encoding and decoding multilayer videos, and more particularly, to encoding and decoding a bit stream of a base layer and a bit stream of a higher layer by encapsulating bit streams in an MPEG-2 Transport Stream (TS) or MPEG-2 Program Stream (PS) packet when the base layer and the higher layer use different codecs in a multilayer video encoding/decoding system.
2. Description of the Related Art
Multilayer video encoding and decoding has been proposed to satisfy many different Qualities of Service (QoSs) determined by various bandwidths of networks, various decoding capabilities of devices, and a user's control. In other words, an encoder generates multilayer video bit streams which are layered by a single encoding, and a decoder decodes the multilayer video bit streams according to its decoding capability. Using this multilayer video encoding and decoding, temporal and spatial Signal-to-Noise Ratio (SNR) layer encoding may be achieved, and encoding and decoding based on two or more layers is also available depending on application scenarios. For example, a base layer encodes videos of a native (or basic) resolution, a first-higher layer performs additional encoding to achieve videos of a higher resolution, and a second-higher layer also performs additional encoding. For reference, a related art multilayer video encoding technology is introduced by the H.262/MPEG-2 system standard.
For a better understanding of aspects of one or more exemplary embodiments, a multiplexing scheme and an MPEG-2 stream of the MPEG-2 system standard will be described below in brief.
Multiplexing in the MPEG-2 system standard is a process of making one Transport Stream (TS) by multiplexing Elementary Streams (ESs) of MPEG-2 video, audio, data, etc., or of multiplexing one or more TSs back into one TS.
FIG. 1 illustrates a multiplexing scheme of the MPEG-2 system standard.
Referring to FIG. 1, video, audio, and data are encoded into associated ESs by a video encoder 101, an audio encoder 103, and a data encoder 105, respectively. The ESs are output as packets by packet generators 111, 113 and 115. The packets are output as a Packetized Elementary Stream (PES) by a program multiplexer (MUX) 120. Multiple PESs are multiplexed into a TS by a transport multiplexer 130.
MPEG-2 streams may be roughly classified into Program Streams (PSs) and TSs. The PS, generated by multiplexing a single program in an error-free channel environment, is a stream to be stored in a storage medium such as, for example, a CD-ROM. The TS, which can be obtained by forming multiple programs in one bit stream as described in conjunction with FIG. 1, is a stream to be transported or broadcasted in the network. For reference, the ‘program’ as used herein may refer to a basic unit including audio/video bit streams.
The TS is identified by a Packet Identifier (PID), and includes a Program Map Table (PMT) and at least one PES. The PMT serves to represent formats and PIDs of program elements constituting one program. The PES is generated by packetizing ESs as described with reference to FIG. 1.
The PS is identified by a stream ID, and includes a PS map and PESs. The PS map has a function similar to that of the PMT.
Bit streams generated for individual layers in the existing multilayer video encoder are considered as one ES because the individual layers use video codecs in the same format as that of the base layer. In other words, as data encoded in a higher layer and data encoded in a lower layer use video codecs in the same structure, both the higher-layer data and the lower-layer data have the same stream ID. For reference, a header of the PES includes a ‘stream ID’ field capable of representing an ID of the stream. In other words, higher-layer data and lower-layer data generated in a multilayer video encoder are treated as one ES, and even in a receiving side, the higher-layer data and lower-layer data are decoded in one decoder as they are considered as one ES. In the following description, the base layer may be represented as a 0th layer, and may be used in the same meaning in a mixed way. The higher layer may include all layers higher than the base layer. However, the higher layers are relative to each other, and, for example, a second layer will be higher than a first layer. Therefore, when 0th, 1st and 2nd layers are provided, the 0th layer is a base layer, and the 1st and 2nd layers are higher layers. Among the higher layers, the 2nd layer is a higher layer with respect to the 1st layer.
If a lower layer and a higher layer generate hierarchical data using encoders having different codecs in multilayer video encoding, an MPEG-2 transport system may not transmit video data that use different codecs in the individual layers. In particular, the MPEG-2 system may not transmit the higher-layer stream when the higher layer uses a codec, for example, a VC-4 codec, other than the standard codecs such as H.264, H.262, MPEG-2, MPEG-4, and VC-4 which are compatible with the MPEG-2 system. This is because the existing MPEG-2 system is assumed to encode the video data using the same codecs in individual layers. Therefore, there is a need for a scheme capable of transmitting encoded data over layers of the MPEG-2 system when individual layers encode data using different codecs in realizing multilayer video encoding.